Spectral Dark Subtraction 

A MODTRAN-Based Algorithm for Estimating Ground 
Reflectance without Atmospheric Information 


Ellis Freedman 

Lockheed Martin Integrated Systems & Solutions 

Robert Ryan 

SSAI 

Mary Pagnutti 

SSAI 

Kara Holekamp 

SSAI 

Gerald Gasser 

Lockheed Martin Stennis Space Center 

David Carver 

Lockheed Martin Stennis Space Center 

Randy Greer 

Lockheed Martin Stennis Space Center 


Agenda 


LOCKHEED M A 



• Background 

• Algorithm Overview 

• Methodology/Approach 

• Results 

• Summary and Conclusions 


i 


Background 
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• In late 1990’s, several systems set requirements for 
determining timely, ground reflectance 

- Several approaches suggested involved commercial, 
government and sounder-based atmospheric data 

• Lockheed Martin took the approach of developing an 
algorithm under internal funding that did not require 
atmospheric knowledge 

• In 2004, LM IS&S enlisted the aid of LM Stennis 
Space Center to enhance and evaluate the algorithm. 
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Algorithm Overview 
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Spectral Dark Subtraction Flowchart 
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Dark Pixel Identification 


• Common Location Histogram 

- Find darkest x% of pixels in each band 

- Determine which pixels are in that portion of all of the 
histograms 

- Works well, but tended to identify “ringing” as dark 
pixels in images that had been oversharpened 

• Normalized Radiance 

- Create the weighted sum of pixel values in all bands 

- Find the pixels with the lowest weighted sum 

- Effectively the darkest “color” pixel 
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Cost Functions 
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• Blue Haze Cost Function 

- Root squared difference between simulated and 
measured slope calculated from blue and red bands 

- Intended to emphasize shape of spectral haze rather 
than absolute match 

• Minimizes impact of correlated calibration errors 

• Dark Dense Vegetation Cost Function 

- RMS of difference between simulated and measured 
values in blue and red bands 

• Shadow Cost Function 

- RMS of difference between simulated and measured 
values in all bands 
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Methodology 


Data Sources Used 
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• LM Stennis, SSAI Stennis, NASA and JACIE 
provided 

- Ikonos imagery and ground truth 

- Quickbird imagery and ground truth 

• Landsat 7 image purchased fromUSGS 

- Ground truth provided by South Dakota State 
University 
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Images Evaluated 


LOCKHEED MARTIN 



Location 

Date 

Sensor 

Sensor Az/EI 

Ground Truth 

Measured 

Visibility 

Brookings, SD 

July 3, 
2001 

IKONOS 

281.8/76.2 

Targets=grass 

ASD, ASR/MFRSR, radiosonde 

49 km ASR 

SSC, MS 

Jan 15, 
2002 

IKONOS 

113.0/77.2 

Targets=3 tarps (3.5, 22, 52), grass, concrete 
ASD, ASR/MFRSR, pressure, radiosonde, BRDF 

316 km ASR 

SSC, MS 

Feb 17, 
2002 

IKONOS 

100.7/81.9 

Targets=3 tarps (3.5, 22, 52), grass, concrete 
ASD, ASR/MFRSR, radiosonde, BRDF 

97 km MFRSR 
148 km ASR 

SSC, MS 

Feb 17, 
2002 

QuickBird 

10.5/67.3 

Targets=3 tarps (3.5, 22, 52), grass, concrete 
ASD, ASR/MFRSR, radiosonde, BRDF 

97 km MFRSR 
148 km ASR 

SSC, MS 

Dec 15, 
2004 

IKONOS 

118.6/68.9 

Targets=3 tarps (3.5, 22, 52), grass, concrete 
ASD, ASR/MFRSR, radiosonde, BRDF 

300 km ASR 

Brookings, SD 

July 20, 
2002 

QuickBird 

349.8/64.1 

Targets=2 tarps (3.5, 52), grass 
ASD, ASR/MFRSR, radiosonde 

26 km MFRSR 
28 km ASR 

Brookings, SD 

Aug 25, 
2002 

QuickBird 

332.5/70.5 

Targets=2 tarps (3.5, 52), grass, concrete 
ASD, ASR/MFRSR 

48 km ASR 

Brookings, SD 

Sept 7, 
2002 

QuickBird 

191.0/74.9 

Targets=2 tarps (3.5, 52), grass, concrete 
ASD, ASR/MFRSR 

48 km MFRSR 
36 km ASR 

SSC, MS 

Nov 14, 
2002 

QuickBird 

274.8/79.4 

Targets=3 tarps (3.5, 22, 52), grass, concrete 
ASD, ASR/MFRSR, pressure, radiosonde, BRDF 

239 km MFRSR 
221 km ASR 
228 km ASR 

Brookings, SD 

Aug 26, 
2003 

Landsat 7 

0/0 

Targets=200m x 1 10m grass site 
ASD, ASR/MFRSR, pressure, temp, RH 

Wx Reported 10 miles 
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Ground Truth 
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• Ikonos and Quickbird 

- Specially designed 20m x 20m tarps 

- Nominal reflectances of 3.5%, 22% and 52% 

- Located at Stennis Space Center and Brookings, South 
Dakota 

- One image of Railroad Valley, NV 

• Landsat 7 

- Carefully measured “grass” field at 3M site near 
Brookings 
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Results 


Landsat Results 
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• Used image of Brookings, SD on August 26, 2003 

- Used a “mowed” section of farm owned by 3M 

- Ground truth provided by South Dakota State 
University 
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Estimated Reflectance 
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IKONOS Band 1 Results 
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IKONOS Stennis Band 1 



Estimated Reflectance 
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IKONOS Band 2 Results 
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IKONOS Stennis Band 2 



Estimated Reflectance 
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IKONOS Band 3 Results 
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IKONOS Stennis Band 3 



Estimated Reflectance 
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IKONOS Band 4 Results 
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IKONOS Stennis Band 4 



Estimated Reflectance 
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QuickBird Band 1 Results 
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QuickBird Stennis Band 1 



Estimated Reflectance 
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QuickBird Band 2 Results 
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QuickBird Stennis Band 2 



Estimated Reflectance 
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QuickBird Band 3 Results 
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QuickBird Stennis Band 3 



Estimated Reflectance 
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QuickBird Band 4 Results 
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QuickBird Stennis Band 4 



Unusual Cases (1) 
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• Requested that LM Stennis find difficult 
atmospheres in which to test the algorithm 

• Provided a collection by Ikonos at Railroad 
Valley, CA with smoke in atmosphere from 
nearby forest fires 
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Railroad Valley Conditions 


University of Arizona Solar Radiometer #2K 
7/13/2002 

Langley Regression Analysis 
Equivalent Visibility: 4.9 km 


Channel (nm) 

Optical Depth 

381 

1.658 

400 

1.557 

440 

1.333 

521 

1.037 

610 

0.832 

671 

0.713 

781 

0.590 

871 

0.486 

940 

1.107 

1030 

0.536 


Smoke significantly reduced 
visibility 

Smoke not visible in imagery due 
to uniformity of ground/target 
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Railroad Valley Results 


Cost 

Function 

Model 

Atmosphere 

Blue Error 

(%) 

Green Error 
(%) 

Red Error 
(%) 

NIR Error 

(%) 

Chosen 
Aerosol Type 

Derived 

Visibility 

(km) 

Blue Haze 

MLS 

Did not 
converge 

Did not 
converge 

Did not 
converge 

Did not 
converge 

Maritime 

2 

Blue Haze 

MLW 

Did not 
converge 

Did not 
converge 

Did not 
converge 

Did not 
converge 

Maritime 

2 

LS Fit 
Blue/Red 

MLS 

-6.1 

-4.0 

-3.1 

0.2 

Maritime 

14 

LS Fit 
Blue/Red 

MLW 

-6.4 

-4.7 

-4.5 

-3 

Tropospheric 

14 

LS Fit 
BGRN 

MLS 

-3.7 

0.7 

1.0 

4.4 

Tropospheric 

11 

LS Fit 
BGRN 

MLW 

-24.5 

1.9 

2.3 

4.3 

Rural 

8.5 
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Unusual Cases (2) 
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• Collection by Quickbird at Brookings, SD on 
7/20/2002 had unusually large errors relative 
to other cases. 
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Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

Qu|ckBi[d_Brookings 07/20/2002 

Blue Band 



7 


QuickBird Brookings 20020720 std MLS PSFON IDnum650 Band 1 



25 


Error 


Reflectance (spherical albedo solution) 


Empirical Line 


Method / Spherical Albedo 

1 LOCKHEED MARTIN 



7 


Method 

QuickBird Brookings 07/20/2002 

Green Band 


QuickBird Brookings 20020720 std MLS PSFON IDnum650 Band 2 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

Qu|ckBi[d_Brookings 07/20/2002 

Red Band 
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QuickBird Brookings 20020720 std MLS PSFON IDnum650 Band 3 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBi[d_Brookings 07/20/2002 

NIR Band 
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QuickBird Brookings 20020720 std MLS PSFON IDnum650 Band 4 
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Error 
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Followup of Poor Brookings Results 


• Originally speculated that the Spectralon reference 
panel used for ground truth was not cleaned properly 

• Contacted Professor David Aaron at South Dakota 
State 

- Responsible for ground truth collections 

• Reference Spectralon panel was clean 

• Atmospheric conditions were marginal to poor 

- Record setting day in Brookings 

• Temperature: 104F 

• Dew point: 77F 

• “The sky had a few cirrus ‘wisps’, but mostly wasn’t really blue. 
Rather it was humid enough so the sky was pushing well 
toward ‘white’ (i.e. just a lot of water vapor scattering).” 


29 


LOCKHEED MARTIN 



Comparison of SDS to Traditional Method 


• NASA sponsored a study of reflectance 
retrieval of the same targets in the same 
images used for the SDS study 

• Retrieval to be performed using MODTRAN 4 
with data from radiosondes launched on-site, 
TOMS (ozone) and MODIS MOD 04 and MOD 
05 products (aerosol and water vapor 
estimates) collected within 20 minutes of 
image collection 

• One to one comparison performed between 
results using both methods 
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Algorithm Results Comparison 


All Targets (52%, 22%, 3.5% Tarps, Grass, Concrete 



— «— Radiosonde/MODIS/TOMS 
— ■ — SDS 


SDS produces more accurate 
results than traditional 
atmospheric data 71% of the 
time against all targets in all 
bands 


52% and 22% T arp T argets 



Trial 


SDS produces more accurate 
results than traditional 
atmospheric data 78% of the 
time against all 22% and 52% 
calibration targets in all bands 
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Conclusions and Summary 


• Spectral Dark Subtraction (SDS) provides good 
ground reflectance estimates across a variety of 
atmospheric conditions with no knowledge of those 
conditions 

• The algorithm may be sensitive to errors from, 

- Stray light 

- Calibration 

- Excessive haze/water vapor 

• SDS seems to provide better estimates than 
traditional algorithms using on-site atmospheric 
measurements much of the time 
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Backup Slides 
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Blue Haze Cost Function 


s - 


R„aJsim,blue) 

1 

3 

[_ R HaJ Sim ’ recl ) 

R ' Haze [meas, red ) _ 


\Y 


-|2 


R Hcre (sim,band) = Haze calculated from MODTRAN 

R' Haze ( meas , band ) = Haze derived from dark pixel characteristics 

R ' Haze ( meas, band) = R dark pixel (band) - p dark _ pixel (band) ■ R m (band) 

R dark pixe i = Measured TOA radiance of the dark pixel 
Pdark _ pixel = Reflectance of the dark pixel 

R wo = Estimated TOA radiance for a 100% reflective target (no atmosphere) 
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Other Cost Functions 


Dark Dense Vegetation RMS Cost Function: 

£ = [ R Haze ( sim, blue ) - R Haze ( meas , blue)] 2 + [R Haze ( sim , red ) - R Haze ( meas , red)] 2 


Shadow RMS Cost Function: 

4 

Haze {sim, band (/)) - R Haze {meas, band (/))] 2 

/=i 
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Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

IKONOS Stennis 02/17/2002 



7 


Blue Band 


IKONOS Stennis 20020217 std atm PSFON IDnum300 Band 1 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

IKONOS Stennis 02/17/2002 



7 


Green Band 


IKONOS Stennis 20020217 std atm PSFON IDnum300 Band 2 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

IKONOS Stennis 02/17/2002 



7 


Red Band 


IKONOS Stennis 20020217 std atm PSFON IDnum300 Band 3 



38 


Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

IKONOS Stennis 02/17/2002 
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NIRBand 


IKONOS Stennis 20020217 std atm PSFON IDnum300 Band A 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

IKONOS Stennis 01/15/2002 
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Blue Band 


IKONOS Stennis 20020115 std atm PSFON IDnum330 Band 1 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

IKONOS Stennis 01/15/2002 
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Green Band 


IKONOS Stennis 20020115 std atm PSFON IDnum330 Band 2 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

IKONOS Stennis 01/15/2002 
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Red Band 


IKONOS Stennis 20020115 std atm PSFON IDnum330 Band 3 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

IKONOS Stennis 01/15/2002 
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NIRBand 


IKONOS Stennis 20020115 std atm PSFON IDnum330 Band A 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBird Stennis 02/17/2002 
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Blue Band 


QuickBird Stennis 20020217 std atm PSFON IDnum400 Band 1 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBird Stennis 02/17/2002 



7 


Green Band 


QuickBird Stennis 20020217 std atm PSFON IDnum400 Band 2 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBird Stennis 02/17/2002 



7 


Red Band 


QuickBird Stennis 20020217 std atm PSFON IDnum400 Band 3 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBird Stennis 02/17/2002 
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NIR Band 


QuickBird Stennis 20020217 std atm PSFON IDnum400 Band 4 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBird Stennis 1 1/14/2002 
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Blue Band 


QuickBird Stennis 20021114 std atm PSFON IDnum430 Band 1 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBird Stennis 1 1/14/2002 



7 


Green Band 


QuickBird Stennis 20021114 std atm PSFON IDnum430 Band 2 



49 


Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBird Stennis 1 1/14/2002 



7 


Red Band 


QuickBird Stennis 20021114 std atm PSFON IDnum430 Band 3 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBird Stennis 1 1/14/2002 



7 


NIR Band 


QuickBird Stennis 20021114 std atm PSFON IDnum430 Band 4 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

Qu|ckBi[d_Brookings 09/07/2002 

Blue Band 



7 


QuickBird Brookings 20020907 std MLS PSFON IDnum730 Band 1 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

Qu|ckBi[d_Brookings 09/07/2002 

Green Band 
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QuickBird Brookings 20020907 std MLS PSFON IDnum730 Band 2 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

Qu|ckBi[d_Brookings 09/07/2002 

Red Band 
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QuickBird Brookings 20020907 std MLS PSFON IDnum730 Band 3 


x 10 

-i10 


Empirical Line Method(ELM) 

♦ Spherical Albedo Method(SAM) 
O EMP Targets 
O SAM Targets 


0.6 


0.4 


0.2 


Error(ERR) 


0.3 0.4 0.5 

Reflectance (linear fit from ASD measurments) 


Optical Depth = 0.0941 
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Error 


Reflectance (spherical albedo solution) 


Empirical Line Method / Spherical Albedo 

1 1 LOCKHEED MARTIN 

Method 

QuickBi[d_Brookings 09/07/2002 

NIR Band 
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QuickBird Brookings 20020907 std MLS PSFON IDnum730 Band 4 
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